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Abstract 

We address the estimation of "extreme" conditional quantiles i.e. when their order 
converges to one as the sample size increases. Conditions on the rate of convergence 
of their order to one are provided to obtain asymptotically Gaussian distributed kernel 
estimators. A Weissman-type estimator and kernel estimators of the conditional tail- 
index are derived, permitting to estimate extreme conditional quantiles of arbitrary order. 



1. Introduction 

Let {Xi, Yi), i = 1, . . . ,n be independent copies of a random pair {X, Y) in E x R. 
where £^ is a metric space associated to a distance d. We address the problem of esti- 
mating q{an\x) G M verifying ¥{Y > q(a„|x)|A = cc) = a„ where a„ — as n — > oo and 
X E. In such a case, g(a„|a;) is referred to as an extreme conditional quantile in con- 
trast to classical conditional quantiles (known as regression quantiles) for which an = a 
is fixed in (0, 1). While the nonparametric estimation of ordinary regression quantiles 
has been extensively studied, see for instance the seminal papers (Roussas, 1969), (Stone, 
1977) or (Ferraty and Vieu, 2006, Chapter 5) less attention has been paid to extreme con- 
ditional quantiles despite their potential interest. Here, we focus on the setting where the 
conditional distribution of Y given X — x has an infinite endpoint and is heavy-tailed, 
an analytical characterization of this property being given in the next section. We show, 
under mild conditions, that extreme conditional quantiles q{an\x) can still be estimated 
through a functional kernel estimator of P(F > .\x). We provide sufficient conditions 
on the rate of convergence of to so that our estimator is asymptotically Gaussian 



distributed. Making use of this, some functional kernel estimators of the conditional 
tail-index arc introduced and a Weissman type estimator (Weissman, 1978) is derived, 
permitting to estimate extreme conditional quantiles q{(3n\x) where /3„ — ?► arbitrarily 
fast. 

2. Notations and assumptions 

The conditional survival function (csf) of Y given X = x is denoted by F{y\x) = 
P{Y > y\X = x). The kernel estimator of F{y\x) is defined for all (a;,?/) e i? x R by 

Fn{v\x) = Y,K{d{x,X,)lh)Q{{Y,^y)/\) / Y,K{d{x,X,) /h), (1) 



1=1 



with Q{t) = q{s)ds where ivT : M+ ^ M+ and q : M -> R+ are two kernel functions, 
and h = hn and A = A„ are two nonrandom sequences such that /i — )■ as n — > cxd. In this 
context, h and A are called window- width. This estimator was considered for instance 
in (Ferraty and Vieu, 2006, page 56). In Theorem 1, the asymptotic distribution of ([Ij 
is established when estimating small tail probabilities, i.e when y — y-n goes to infinity 
with the sample size n. Similarly, the kernel estimators of conditional quantiles q{a\x) 
are defined via the generalized inverse of Fn{.\x): 

qrMx) = inf{t, FrXAx) < a}, (2) 

for all a € (0, 1). Many authors are interested in this type of estimator for fixed a G (0, 1): 
weak and strong consistency are proved respectively in (Stone, 1977) and (Gannoun, 
1990), asymptotic normality being established when E is finite dimensional by (Stute, 
1986), (Samanta, 1989), (Berlinet et al., 2001) and by (Ferraty et al, 2005) when E is 
a general metric space. In Theorem 2, the asymptotic distribution of ([2]) is investigated 
when estimating extreme quantiles, i.e when a = a„ goes to as the sample size n goes 
to infinity. The asymptotic behavior of such estimators depends on the nature of the 
conditional distribution tail. In this paper, we focus on heavy tails. More specifically, 
we assume that the csf satisfies 



(Al): F{y\x) = c{x) exp |- - s{u\x)^ ^ 



where 7(.) is a positive function of the covariate a;, c(.) is a positive function and |e(.|2:)| is 
continuous and ultimately decreasing to 0. (Al) implies that the conditional distribution 
of Y given X = x is in the Frechet maximum domain of attraction. In this context, 
7(2;) is referred to as the conditional tail-index since it tunes the tail heaviness of the 
conditional distribution of Y given X = x. Assumption (Al) also yields that F{.\x) is 
regularly varying at infinity with index —1/7(2;). i.e for all C > 0, 

hni ^(Cy|x)^^_i/,(.)^ 
J/-S-00 Flylx) 

The function £(.|x) plays an important role in extreme- value theory since it drives the 
speed of convergence in ([3]) and more generally the bias of extreme-value estimators. 



Therefore, it may be of interest to specify how it converges to 0. In (Gomes et ai, 
2000), the auxihary function is supposed to be regularly varying and the estimation of 
the corresponding regular variation index is addressed. Some Lipschitz conditions are 
also required: 

(A2): There exist k^, Kc, > and wo > 1 such that for all {x, x') G E'^ and u > uo, 

|logc(a;) — logc(a;')| < Kcd{x,x'), 
\e{u\x) — s{u\x')\ < Kgd{x,x'), 
|l/7(ar) - l/7(a;')| < n^d{x,x'). 

The last assumptions are standard in the kernel estimation framework. 

(A3): if is a function with support [0, 1] and there exist < Ci < C2 < cxd such that 
Ci < K{t) < C2 for all t e [0, 1]. 

(A4): g is a probability density function (pdf) with support [—1, 1]. 

One may also assume without loss of generality that K integrates to one. In this case, 
K is called a type I kernel, see (Ferraty and Vieu, 2006, Definition 4.1). Finally, let 
B{x, h) be the ball of center x and radius h. The small ball probability of X is defined 
by Vx{h) = '¥{X e B{x,h)). Under (A3), for all r > 0, the rth moment is defined by 
tx^j\h)=E{K^d{x,X)/h)}. 

3. Main results 

Let us first focus on the estimation of small tail probabilities F{yn\x) when t/„ — ^ 00 
as n — >■ 00. The following result provides sufiicient conditions for the asymptotic normal- 
ity of Fn{yn\x). 

Theorem 1. Suppose (Al) (A4) hold. Let x ^ E such that ipx{h) > and introduce 
i/n.j — OjUn for j = 1, . . . , J with < ai < a2 < ■ ■ ■ < aj and where J is a positive 
integer. If yn — > 00 such thai nipx{h)F{yn\x) 00, rupx{h)F{yn\x){X/yn)^ — >■ and 
n^x{h)F{yn\x){hlogyn}^ as n 00, then 



is asymptotically Gaussian, centered, with covariance matrix C{x) where Cjj'{x) = 
a]ip for{j,r)G{l,...,Jr. 

Note that n(px{h)F{yn\x) — >■ cxd is a necessary and sufficient condition for the almost sure 
presence of at least one sample point in the region B{x,h) x (j/„,oo) of E xM.. Thus, 
this natural condition states that one cannot estimate small tail probabilities out of the 
sample using F„. This result may be compared to (Einmahl, 1990) which establishes 
the asymptotic behavior of the empirical survival function in the unconditional case but 



without assumption on the distribution. Letting (7n{x) = {nii2'\h)an)~^^'^ , the asymp- 
totic normahty of qn{ctn\x) when a„ — >■ as n — >■ cx) can be estabhshed under similar 
conditions. 



j=i,...,j 



Theorem 2. Suppose (Al) - (A4) hold. Let x € E such that <Px{h) > and introduce 
oin,o = '''jOtn for j = 1, . . . , J With Ti > T2 > . . . > T j > ttud wherc J is a positive 
integer. If an ^ such that — >■ 0, a~^{x)X/q{an\x) — >■ and a~^{x)hlogan — >■ 
as n —i- 00, then 

V <l{an,j\x) 

is asymptotically Gaussian, centered, with covariance matrix 'y'^{x)T, where Sj j/ = I/tjaj' 
for {!,..., jy. 

The functional kernel estimator of extreme quantiles (7„(a„|a;) requires a stringent condi- 
tion on the order q:„ of the quantile, since by construction it cannot extrapolate beyond 
the maximum observation in the ball B{x, h). To overcome this limitation, a Weissman 
type estimator (Weissman, 1978) can be derived: 

C(^nk) = 9nKk)K//3„r''(-). 

Here, qn{ctn\x) is the functional kernel estimator of the extreme quantile considered so 

far and ^n{x) is a functional estimator of the conditional tail-index 7(.x). As illustrated 
in the next theorem, the extrapolation factor {oin / l^nV"^^^ allows to estimate extreme 
quantiles of order /?„ arbitrary small. 

Theorem 3. Suppose (Al) (A^) hold. Let us introduce 

• a„ such that <Jn{x) 0, cr^^ (a;)A/2/„ — >■ and a~^(a;)ft,loga„ — )■ as n ^ oo, 

• {fin) such that j3n/cin —^0 as n ^ CO, 

• 7n(a;) such that G~^{x){^n{x) —^{x)) J\f{0,v'^{x)) where v'^{x) > 0. 
Then, for all x G E, 

l0g{an/Pn)[q{f}n\x) ^J^^^i^^^Wh 



Note that, when K is the pdf of the uniform distribution, this result is consistent with 
(Gardes et al., 2010, Theorem 3), obtained in a fixed-design setting. 

Let us now give some examples of functional estimators of the conditional tail-index. Let 
a„ — > and ti > T2 > . . . > tj > where J is a positive integer. Two additionary 
notations are introduced for the sake of simplicity: u = (!,...,!)* S R"^ and v = 
(log(l/ri), . . . , log(l/Tj))* G M-^. The following family of estimators is proposed 

. . ^ (p{\ogq„XTi(\„]j:), . . . .log q„{Tj(\„]x)) 
^"^''^ <^(log(l/ri),...,log(l/T^)) 



where (f : R'^ M. denotes a twice diffcrcntiable function verifying the shift and location 
invariance conditions (p{9v) = 9(p{v) for all > and (p{r]u + x) = (p{x) for all G M and 
X e M''. For instance, introducing the auxiliary function mp{xi,. . . , xj) — J2^=i{^j ~ 
XiY for all p > and considering ifnix) = mi {x) gives rise to a kernel version of the Hill 
estimator (Hih, 1975): 

,7 / J 

7n(2;) = [log9"(r,Q:„|a;) - logq„(a„|x)] / ^log(l/rj) . 
j=i I j=i 

Generalizations of the kernel Hill estimator can be obtained with <p{x) = nip (x) /mf ^ ^ (x) , 
see (Gomes and Martins, 2001, equation (2.2)), ip{x) = rr^/^{x), see e.g. (Segers, 2001, 

1 / 

example (a)) or Lp{x) = mjg {x)/'mp^i{x), p > 1, 6* > 0, see (Caeiro and Gomes, 2002). 
In the case where J = 3, ti = 4, r2 = 2 and T3 = 1, the function 

'expa:2 — expa;i ' 



ipp{xi,X2,X3) = log 

\expx3 — expx2, 
leads us to a kernel version of Pickands estimator (Pickands, 1975) 
.p. ^ 1 , / qnia„\x) - qn{2an\x) 

7n(2^) = T— T^log 



log 2 \qn{2an\x) - g„(4a„|.T) 

We refer to (Gijbels and Peng, 2000) for a different variant of Pickands estimator in the 
context where the distribution of Y given X = x has a finite endpoint. The asymptotic 
normality of ■Jn{x) is a consequence of Theorem 2. 

Theorem 4. Under assumptions of Theorem 2 and if a~^{x)e{q{Tian\x)\x) — )• as 
n — >■ 00, then, a~^{x){^n{x) — 7(2;)) converges to a centered Gaussian random variable 
with variance 

V{x) = ^(V^(7(x)^;))*E(V^(7(x)^;)). 

As an illustration, in the case of the kernel Hill and Pickands estimators, we obtain 

2 

v^{x) = i\x)\y y'" ^'- -j'^ ^ 





V^{x) 



7^(a;)(2^T(^)+i + l) 
4(log2)2(2-r(^) ^ 1)2 



Clearly, Vp{x) is the variance of the classical Pickands estimator, see for instance (de 
Haan and Ferreira, 2006, Theorem 3.3.5). Focusing on the kernel Hill estimator and 
choosing = 1/j for each j = 1, J yields V^{x) = 72(3;) J(J- 1)(2 J- 1)/(6 log^( J!)). 
In this case, Vii{x) is a convex function of J and is minimum for J = 9 leading to 
V^{x) ^ 1.2572(0;). 
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